DOCOMENT RESUME 



ED 084 661 



EA 005 6a0 



TITLt' 

INSTITUTION 

PUB DATE 
NOTE 

LDES PRICE 
DESCPIPTOBS 



Comprehensive Planniny Model for Educational 
Programs. 

Nevada Stata Dept. of Education^ Carson City, Div, of 
Planning and Evaluation, 
Aug 73 
25f, 

aF-$0,65 HC-$3,29 

Decision ^?aking; Educational Objectives; ^Educational 
Planning; ♦Educational Programs; *Models; Performance 
Criteria; Problem Solving; ^Program Evaluation; 



IDENTIFIERS 



Program Planning; Public 
Departments of Education; 
Approach 

Needs Assessment; Nevada 



hool Systems; ♦State 
tate Programs; Systems 



ABSTRACT 

The existing crises in the scho 
some systematic method of planning tliat leaves 
opinion and which rests more on the scientific 
solving. Although the use of a systems approach 
just beginning, it seems to hold considerable p 
Hence, the Nevada State Department of Education 
for systematic, comprehensive educational progr 
evaluation which should be applicable to educat 
level of operation or complexity. The model pro 
steps for a systematic approach to problem solv 
problem definition, program development, progra 
evaluation, final analysis of evaluative data, 
recycle. The approach is predicated on the exis 
relationship between goals and objectives. (Aut 



ols show a nee 
little to chan 
techniques of 

in public pro 
romise for edu 

has designed 
am planning an 
ional problems 
vides seven ba 
ing: needs ass 
m operation an 
dissemination, 
fence of a wor 
hor/WM) 



d for 
ce or 
problem 
grams is 
cation, 
a model 
d 

at any 
sic 

essment , 
d 

and 
kable 



FILMED FROM BEST AVAII.ADLE COPY 




COMPREHENSIVE 
PLANNING MODEL 
FOR EDUCATIONAL 
PROGRAMS 

August, 1973 




Kenneth H. Hansen 
Superintendent of Public Instruction 



State Board of Education 

MEMBERS 

Robert I. Rose, President, Reno 
Rosemary K. Clarke, Vice President, Las Vegas 
Cynthia W. Cunningham, Las Vegas 
The Rev. Benjamin J. Franzinelli, Las Vegas 
Dr. Mary H. Fulstone, Smith 
George E. Harris, Las Vegas 
Joan Kenney, Las Vegas 
William R. Orr, Pioche 
Shirlee a. Wedow, Sparks 



DIVISION OF PLANNING AND EVALUATION 

Jiimes Kiley, Associate Superintendent 
Dr. Kay W. Palmer, EMIS Director 
Dr. R. H. Mathers, Consultant, Assessment and Evaluation 
Dr. Austin Haddock, Consultant, Planning 



NEVADA STATE DEPARTMENT OF EDUCATION 
Carson City. Nevada 



ACKNOWLEDGMENTS 



Appreciation is due Dr. Austin E, Haddock for the background research and 
development of the basic content of this document and to Mr, James P. Kiley, Dr, 
Kay W. Palmer, and Dr. R. H. Mathers of the Division of Planning and Evaluation, 
for their conscientious evaluation of the planning model. 

To James H. Bean, James Menath, Ron Gutzman, Denis Graham, Jerry Nielsen, 
William Arensdorf, and Gene Robinson, for their critical review of the format and 
content, special thanks for the time and energy diverted from busy schedules to this 
activity. 

Acknowledgment is given to the entire Department of Education staff for their 
patience and wholehearted support, not only in advising and assisting in the devel- 
opment of the system, but in applying the concepts to their daily operations. 

If support for the implementation of the concepts embodied within this model 
continues, we may expect a significant breakthrough in educational planning and 
accountability. 

Kenneth H. Hansen 
Superintendent of 
Public Instruction 



ERLC 



V 



PREFACE 



The purpose of this document is to provide information about planning and eval- 
uation within the Nevada Department of Education and to present a comprehensive 
planning model which may be applied at any level of operation in any enterprise. 

It often takes twice as long to plan a project, for example, San Francisco's 
"Golden Gate," as it takes to construct it. Few would suggest that the operation of 
our social institutions is less complex than building a bridge, yet we operate for 
years on plans which are sometimes made overnight, or on old plans that havo never 
been evaluated. 

Educational theorists have paid only passing attention to the problem oi translat- 
ing their findings to the classroom situation. Methodologists usually focus on the 
"how to" aspects of operation. The Division of Planning and Evaluation was cre- 
ated by the Nevada Department of Education to help bridge the gpp between theory 
and operation, to facilitate planning and to provide the means by which the ade- 
quacy of planning and operation can be evaluated. 

The existing crises in the schools show that some systematic method of planning 
is needed which leaves little to chance or opinion and rests more upon the scientific 
techniques of problem solving. In the midst of every endeavor is the task of making 
theory and practice mutually consistent. Education is no exception, and, as in any 
other long-established institution, there is the ever present danger of operating by 
theoretical conclusions without submitting them to practical test. 

Much of teaching and education is a creative process, but in the rush of day-to- 
day activities we may forget the importance of refining our thoughts so Ihey become 
relevant to the real world. We need a system which is compatible in theory and in 
operation and which has the inherent flexibility for general and specific application. 
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FOREWORD 



Increased federal participation^ in state and local educational programs, accel- 
erated by Public Law 89-10 in 1965, brought heightened awareness of the need for 
definitive planning and evaluation. In 1968, the Nevada Governor's State Council 
on Vocational-Technical Education Community Colleges charged the SlAc 
Department of Education . . with the responsibility of developing a master plan 
for education in the State of Nevada. . . In addition, the 1968 Special Session of 
the Nevada Legislature charged the State Department of Education as follows: 
'*The State Department of Education shall study and recommend to the 55ih Ses- 
sion of the Nevada Legislature a master plan for education. . . The master plan 
was completed and published in 1969.* 

In March 1969, the Department negotiated a contract with the Research and 
Educational Planning Center of the University of Nevada at Reno for a statewide 
assessment of education. The first phase (statistical information) was completed 
and published May 1, 1969. In 1969-70 the second phase (a statewide educational 
assessment) was completed and published. ^ 

On June 5, 1970, the U.S. Office of Education granted funds for the establish- 
ment of a planning and evaluation division within the Nevada State Department of 
Education. On November 2, 1970, the division was activated with the following 
goal; 'To design, develop and implement a planning and evaluation unit in a man- 
ner that will provide continuous maximum service to the SEA and to the LEA\s 
and will result in the improvement of public education throughout the State.'' 

Although program planning has always been a function of the Nevada State 
Department of Education, it has never been conducted on a systemaiic and com- 
prehensive basis. Further, program evaluation has not been coordinated and pro- 
gram information has been gathered on a piecemeal basis. A further complication 
has been the fact that information has not always been retrievable in a form that 
could serve the needs of program planners and managers. 

Although evaluation occurs at the level of program management, the degree of 
formality and the form varies considerably among programs. One consequence is 
that the interfacing of information from several sources has been the result of a 
great deal of ad hoc effort rather than a systematic organizational function. 

On January 4, 1971, the Department of Education Cabinet adopted ten *'Com- 
mon Goals of Nevada Education" and instructed the Division of Planning and 
Evaluation to expedite a procedure for development of process and performance 
objectives which would set in motion some systematic progress toward the goals. 
The goals are: (1) fostering creativity, (2) vocational productivity, (3) continu- 
ing education, (4) intergroup acceptance, (5) motivation to learn, (6) citizenship 
and social acceptance, (7) self-understanding and acceptance, (8) mastery of basic 
skilfs (9) physical and emotional health and (10) intellectual development.^ 

Nevada's common goals are arranged in a hierarchy in three areas of basic 
human needs: physiological and safety; aflfiliation and esteem; and self-actualization, 
cognitive and aesthetic. These goals also presuppose that the overriding purpose of 
education is to increase the functional and self-actualizing abilities of people. 

^State Department of Education. Planning Education for Nevada's Growth. Carson City: 
The Department, 1969. Two volumes.) 

^State Department of Education. Education in Nevada, An Assessment for 1970. Carson 
City: The Department, 1970. 

^Common Goals of Nevada Education. Carson City: State Department of Education, 1961. 
16 pages. 
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RATIONALE 



Faced with complex and often conflicting demands, 
plus a scarcity of resources to meet all the urgent needs, 
the Nevada State Department of Fducation concluded 
that some nuthod should be designed which would pro- 
vide a systcmaiic procedure for establishing priorities, 
attacking problems and evaluating outcomes. Originally 
developed in the national defense and space elTorts, the 
systems approach has spread through private and gov- 
ernmental agencies. Although the use of systems prob- 
lem solving in public programs is just beginning, it seems 
to hold considerable promise for education: hence, the 
Departm^fnt's Division of Planning and Evaluation has 
designed 'a model for .systematic, comprehensive educa- 
tiofhil pr(\qram planning; and evaluation. 

WHAT IS THE SYSTEMS APPROACH? 

The systems approach is the scientific method 



applied to the solution of problems. It places emphasis 
on identifying, defining, analyzing, s>ntlicsi/jng, hypoth- 
esizing, programing, monitoring, and evaluating. This 
approach seeks the best alternative for achieving speci- 
fied objectives. \\ hile this approach is only one part of 
policy d'X'ision-niaking, it provides a n«nv capability in 
what is often a crucial area of decision. The traditional, 
scientific approach, from which the systems approach 
derives, included the general steps of ( I ) identifying the 
problem, (2) establishing an hypothesis, (3) testing 
and rrtcsting, (4) evaluating, and (5) establishing a 
working conclusion W be retested and refined. Present 
day models for problem solving vary in their degree of 
emphasis and refinement of the various steps in the 
scientific method, but they all rely upon some system- 
atic approach to identification of needs, p'^oblcm defini- 
tion, solution strategy, operation, and evaluation. 



A MODEL FOR PLANNING 



The Nevada Department of Education has prepared 
a general model, which should be applicable to educa- 
tional problems at any level of operation or complexity. 



The model (sec Figure I) provides seven basic steps 
for a systematic approach to problem solving. 

Each of these steps may be broken into several 
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substcps as shown in subsequent sections of this docu- 
ment. Key factors to keep in mind when using the model 
include the degree to which the situation to be changed 
is under the control of the agency. It is also important 
to keep in mind that the determination of which prob- 
lem or problems will receive priority should be made by 
appropriate policy makers. Evaluation is a continuous 
clement. 

A systematic, controllable approach to planning and 
evaluation is predicated upon a workable relationship 
between goals and objectives. Planning, operation, eval- 
uation, and accountability presume goals and objectives. 
The more clearly goals and objectives are stated and 
related, the more efTective the total operation will be. 
They should be the major basis of assessment, planning." 
operation, and evaluation (see Appendix B). 

Goals are defined as ends toward which an agency 



or person performs work or renders services. Objectives 
are the intermediate ends for which work is performed 
or services rendered; the desired product of an activity 
representing progress toward goals. 

The Policy Manual of the Nevada State Board of 
Education includes the following statements: 

It is the responsibility of the State Board of 
Education to help promote a system of education 
that will fit the needs of a highly diversified citizen- 
ship at the most economical rate of expenditure. It 
is the responsibility of this Board to interpret to 
the best of its ability to the Legislature, the educa- 
tional needs of the State and to make its interpre- 
tations known to the public. In this responsibility, 
it shall need always the counsel of local school 
boards and citizens interested in educational wel- 
fare.^ 



NEEDS ASSESSMENT 
(Step 1.0) 



Perhaps the most tedious and challenging step in 
needs assessment is delimiting the priority need. There 
is a strong tendency for policy makers to start with a 
need which is too broad and general, such as to increase 
the rate of growth in reading achievem^^nt for all pupils 
K-12. While such an effort is commendable, it is difli- 
cult and expensive to monitor and control in explicit 
fashion. If such broad need can be approached by an 
in-depth analysis of needs in reading for 9-year-old chil- 
dren however, then it becomes more feasible. Many of 
the findings can be related to continuing assessment and 
problem solving at other age levels. 

Delineate the need in detail before applying broad 
solution strategies. The overall concept of needs assess- 
ment advocates a routine, formalized, and on-going 
process to determine whether or not there are discrep- 
ancies between what the policy makers believe ought to 
be and what is. It helps effect communications in plan- 
ning and programming to establish a list of priority 
needs in terms of their criticality, frequency, and their 
probable short-range and long-range consequences. It is 
also important to consider time, resources, and political 
realities relating to priority needs. 

Determination of the need(s) to receive high priority 
is a responsibility of management. In making this deter- 
mination, a conceptualization of values and objectives 
in the area(s) under study is pertinent. A skeletal out- 
line of such a conceptual relationship is presented in 
Figure 2. 

A hierarchy is involved in the sense that societal val- 
ues strongly influence decisions made at the institutional 
and instructional levels. 

It is a reasonable assumption that any human 
endeavor is guided by more or less well-defined goals, 

'Policy adopted, State Board of Education, 1964 (see Policy 
Manual). 



and objectives. The degree to which we perceive the 
hierarchical order determines, to a great extent, the 
practicality with which we prioritize our needs. 

In the daily operation of the educational institutions, 
the interested parties, e.g. society, institutions, and 
teachers, have goals and some more or less well- 
conceived objectives leading toward the goals. An 
important part of" a needs assessment is. therefore, a 
recognition of the hierarchy of goals and objectives as 
it relates to societal expectations, to institutional goals 
and to instructional objectives. The difference between 
status (what is) and the objective (what should he) is the 
need. 

In the needs assessment step of program planning, we 
must establish learner performance objectives before we 
can analyze the need (difference between what is and 
what should be) and establish some estimate of the 
severity of need. Needs assessment, step one in the 
model for comprehensive educational program plan- 
ning, is depicted by Figure 3. 

Societal expectations, albeit very important, are often 
somewhat vague and elusive. Periodically, educational 
institutions should conduct a survey and aniysis of the 
community expectations to get some "fix" on the way 
society perceives needs. 

Suppose that a perceptual survey indicates that third- 
year elementary pupils are reading below expectation. 
Further assume that the level at which they should be 
reading has been agreed upon. Try then to determine, 
as accurately as possible, just how much below expecta- 
tion they are reading (the status of pupil performance). 
Using all available information, determine which chil- 
dren are achieving below expectation and how much 
(and which are achieving above); what ethnic socio- 
economic levels, low I.Q,, health, attendance, promo- 
tions, retentions, grades, handicaps, self-concept, and 
aspiration factors may be involved; in short, collect as 
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much specific data as possible. It is important to know 
ihc severity of the discrepancy as related to the sehiH)l 
and societal experiences students ha\e h 1. 

At this le\cl of needs assessment a great deal of lime 
and elTorl may be involved in order to establish state 
and or district objectives and then to determine just 
where children are in relation to those objectives. When 
th*s is done in the designated area, more specific pupil 
pcrfonnofuc ithja uvcs and specific process ohjcaivcs 
siunild be wrillen, (Sec Appendix B for definitions of 
objcclivcs.) It then becomes possible to prioritize needs, 
develop specific programs and evaluate results acci>rd- 
ing to a hierarchy of specific goals and objectives. The 
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PROBLEM DEFINITION 
(Step 2.0) 



*'Thc formulation of a problem is , , , often [more| 
essential than its solution, which may be merely a mat- 
ter of mathematical or experimental skill," — Albert 
i:instein. 

S>ep two in planning, a vital step in the analysis of 
nevd, may be outlined as in Figure 4, 

A problem is herein defined as the requirement for a 
strategy or means to reduce or eliminate a need (the 
difference between what is and what should be). Prob- 
lem definition is a delineation of the factris and influ- 
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according to specifications and should enable managers 
to make judgments about their effectiveness. 

Four potential alterr itives which mast be considered 
as a result of such analysis are: 

1 . Level of performance saiijifactory — reinvest 
resources in previous processes. (The problem is to 
maintain the performance level. The need was not as 
f.real as first perceived.) 

2 Level of performance not attained — processes 
remain the same (performance expectations too high). 
Change performance objectives. 

3. Level of performance not attained — reinforce 
management of processes and reinvest resources (same 
or additional). 

4. Level of performance not attained — new proc- 
esses developed. 

Problem definition involves a restatement of the pri- 
ority need in terms of its related aspects, e.g., reading 
vocabulary, word attack skills, comprehension, and like 
factors: break it down by sub-groups, i.e., ethnic 
groups, age levels, region, or districts; finally, bring all 
the available information to bear in establishing work- 
ing conclusions as to the whys. In short, a carefully 
delineated restatement (in terms of new information 
and insights) of the problem is in order. A suggested 
list of solution processes would be helpful at this point 
since the consideration for revision of process objectives 
will be critical in program development. 

VARIABLES AFFECTING A PROBLEM 

In defining the problem, we define those who are to 
perform the desired behavior (institutional dimension), 
what behavior is desired (behavioral dimension) and 
under what circumstances or conditions the behavior 
will be observed (instructional dimension). Since all the 
variables under each of these three dimensions have 



important relationships to the educational program, 
should consider then\ collectively and separately. 

Institutional Variahles\ When striving to increase 
gains in reading skills of third-graders the n.ajor institu- 
tional variable is third-gr:ide students, ft is relevant that 
characteristics of the group be considered, i.e.. age. sex. 
achievement levels, ethnic background, health, self- 
concept, and attitudes. 

Consider other institutional variables such as te:ich- 
ers* experience, training, attitudes, and special abilities 
as well as the specialists required for unique instruc- 
tional settings, i.e., remedial teacher, eurricuUnn coor- 
dinator, and teacher aides. F^amily situations. <'onimunity 
'ntluence, institutions (religious, political, ehibs. etc.). 
and peer groups are also relative variables. 

Behavioral Variables. Though the majority of vari- 
ables may be in the cognitive domain, do not ignore the 
relative influence of afTeetive and psychomotor vari- 
ables. Decide what level of proficiency is desired, in 
what specified period of time, and decide what measure- 
ment instruments will be used. A committee should be 
appointed to review measurement instruments in terms 
of their adequacy for the levels of beiiavior (knowledge, 
comprehension, application, analysis, synthesis, or 
evaluation in the cognitive domain, for example), and 
to select the most appropriate instruments. These evalu- 
ation instruments should be chosen on the basis of the 
pcrfo^'manee objectives and as a part of the evaluation 
design. 

Since standardised measurement instruments never 
measure all the learner skills in a particular subject 
area, it is imperative that i;cm pools be developed for 
future program uses. From such pools, criterion- 
referenced tests may be constructed which will measure 
learner performance more reliably. 

Instructional Variables. Organization, content, 
method, types of interaction, facilities, and costs should 
be determined in this area. 



PROGRAM DEVELOPMENT 
(Step 3.0) 



Figure 5 indicates the interrelated substeps in pro- 
gram development. 

PERFORMANCE OBJECTIVES 

Having analyzed the variables in connection with 
time and proficiency level, specific performance objec- 
tives should now be written. For example: At the com- 
pletion of 36 weeks of school (time) third-grade 
students (institutional variable) will show an increase 
of 9.5 months (proficiency level) in reading (instruc- 
tional variable content) comprehension (behavior var- 
iable) as measured by a standardized achievement test 
(measurement). This is a school objective. Classroom 
performance objectives should also be written, on a 



more definitive basis, according to the ability level of 
the group. 

The testing committee can now make preliminary 
plans for selecting measurement instruments. (See 
Appendix B.) 

PROGRAM PLANS AND SPECIFICATIONS 

Alternative Solutions. Having developed more spe- 
cific and comprehensive performance objectives, we 
must consider ahernative solution strategies and pos- 
sible changes in the variables, including cost. These 
alternatives should be described in writing for purposes 
of comparison by the decision makers. 

Select Best Solution. Using all the available docu- 
mentation relating to each possible solution, we should 
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explore the feasibility and potential of each alternative. 
The optimum choice should be made in terms of physi- 
cal and political realities as they relate to school orga- 
nization, methods, equipment, facilities, time, personnel, 
cost-effect, and other real-world parameters. Program 
plans and specifications should be developed and writ- 
ten in detail, considering such items as learner selection 
and placement, performance objectives, curriculum, 
instruction, staff role and function, evaluation strategies 
and design, data collection, reporting formats, calendar 
of events, and dissemination of information, 

Staff role and function specifications include such 
areas as management, instructional services, audit 
points, staff training (including writing process objec- 
tives), and auxiliary services. These specifications are 
important to the "best" solution strategy. Written rec- 
ord?; of these specifications should be maintained. 

Process Objectives. (See Appendix B.) A process 
objective is an activity conducted by a staff member, the 
completion of which leads toward the accomplishment 
of a learner performance objective. For example: 

The third-grade teacher (person responsible) 
will review pupil's readiness (activity) from cumu- 
lative records, including diagnostic tests, and 
assign him to appropriate reading group (tangible 
outcome) by October 10 (time factor); fifteen 
minutes per pupil (cost). \ 

The above is a process objective of a third-grade 
teacher intended to lead to a performance objective 
such as: 

Third-grade students (institutional variable) 
will increase their level of comprehension (behav- 
ioral variable) in reading (instructional variable) 
by 10 months (proficiency level) by June 1 (time) 
as measured by the Comprehensive Test of Basic 
Skills (method of measurement). 

In any instructional program, process objectives 
should relate to the performance objectives of the 
instructional program. Process objectives should be 
written by or for each professional staff member for 
each new or revised program. 

Modification of process objectives may be required at 
any time in the program operation; consideration for 
revision is critical at the program planning stage. Proc- 
ess objectives should be written as clearly and defini- 
tively as possible before a new or revised program is 



launched, with the performance objectives uppermost in 
mind. 

Evaluation Design. (See Figure 8 for evaluation 
model,) Evaluation is a continuous process which starts 
with needs assessment, proceeds through progt^iin 
development and operation, through the measurement of 
objective attainment, and returns again, in the recycling 
piocess, to subsequent needs assessments. Evaluation 
must have performance objectives as its base, otherwise 
there is nothing to evaluate. No educational program 
can be evaluated unless the performance objectives of 
that program are specifically stated (see performance 
objectives. Appendix B). Only in this way can any logi- 
cal measurement design be developed; only in this way 
can the instructional components of the program be 
rationally monitored; and only in this way can recom- 
mendations for program change and hence learner 
change be determined, 

Calendar of Events. A written calendar of events 
has value for communications, work flow, monitoring 
and evaluation. It may be in tabular form or a work- 
flow chart which may be wall-mounted for easy refer- 
ence. 

While important dates are noted throughout the pro- 
gram development step, it is not until they are placed in 
some chronological order that potential conflicts can be 
resolved. Among others, completion dates for learner 
selection, staff training, test selection, guidelines and 
specifications for instruction, testing dates, statistical 
processing, final reports and dissemination dates should 
be calendared. Completion dates for process objectives 
should be listed sequentially. 

Program Budget. At this point, prepare an alloca- 
tion of resources (people, time, money, facilities and 
equipment) and an estimated budget (use process 
objectives to derive cost estimates related to perform- 
ance objectives); submit the total package to authorities 
for approval. 

ASSIGNMENT OF RESOURCES 

Having gained final approval (with any modifica- 
tions), the final step, before commencing operations, is 
a studied assignment of resources with whatever staff 
and/or committee meetings seem appropriate to secure 
maximum understanding and commitment to the pro- 
gram. 



PROGRAM OPERATION AND EVALUATION 

(Step 4.0) 



Frequently this step would involve a pilot or field 
trial of the plan. If plans have been made in sufficient 
detail, and personnel and other resources allocated 
wisely, the program is ready for implementation. 

It is vital to the success of the program to develop a 
monitoring system for both process and performance 
objectives to assure that all phases of the operation are 



on target and to process any necessary modifications. A 
system for monitoring progress via the calendar of 
events, and for collecting and processing data, should 
have been a staff-designated responsibility for which 
process objectives were written. These vital activities 
are a part of the formative evaluation. 

Figure 6 indicates the major areas of consideration 
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FIGURE 6: Program Operation and Evaluation 



during program operation. In the evaluation model (sec 
Figure 8), this is the implementation phase of evalua- 
tion. 

Any necessary variations in the way pupils are orga- 
nized in the classroom or variations in content, method, 
or facilities should be noted for future reference and 
reporting. One purpose in monitoring is to control the 
variables in a prudent fashion. 

Cost records should be maintained as accurately as is 
practical for accounting purposes. 



Collecting interim measurements may consist of 
criterion-referenced tests (from an item pool), teacher- 
made tests and other planned instruments. 

While good planning should provide the basic para- 
meters, anything as important as evaluation should 
always be open to improvements from new insights, 
hence review and revise as necessary. 

No person or persons can foresee all pertinent activ- 
ities; hence the calendar of events review and revision 
is always appropriate. 



ANALYSIS OF EVALUATIVE DATA 
(Step 5.0) 



The major considerations in the analytic stage of 
evaluation are outlined in Figure 7. These steps are 
listed under Measurement of Objective Attainment in 
the evaluation model. (See Figure 8.) 

In this stage of the evaluative process (the summative 
stage), post-program data are analyzed and compared 
with pre-program data, learner attainment is measured 



against performance objectives, and recommendations 
are made for use of these analyses. Attempts should be 
made to determine the extent to which the objectives 
were achieved (or not achieved) as attributable to the 
planned program. Where possible, new program out- 
comes should be compared to previous program out- 
comes> 
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DISSEMINATION 
(Step 6.0) 



Dissemination of pertinent information (including 
recommendations for program changes) to appropriate 
publics is an important and often poorly implemented 
phase of the operation. Specifications should be drawn 
up in the planning stage by a task force and reviewed 
for adequacy during the operation and before release. A 
written report to the decision makers should include 
precise information and interpretations supported by 
explanations for any modifications which occurred dur- 
ing operation. Recommendations and the bases thereof 
for further modification and continuation of the pro- 
gram are appropriate. (See Figure 9.) 

Certain information will be channeled to or through 



the nistitution's Educational Management Information 
System (EMIS) for analysis and synthesis. The EMIS 
has a storage and retrieval function vital to future pro- 
gramming as well as to programs in progress. Sonic 
information will be subject lo dissemination upon short 
notice because of its timeliness. Certain information 
should be fed back to participants throughout thj prog- 
ress of the seven basic steps outlined in the planning 
and evaluation model. 

Dissemination to interested publics of categorized 
findings, at the end of a cycle, is a vital process which 
should be determined by a task force in collaboration 
with the EMIS director. Figure 9 suggests a possible 
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flow of information. The model is subject to consider- 
able elaboration or modification in terms of conmiittee 
and task force assignments and reports. 

So far as program evaluation is concerned, the dis- 
semination of data should contain the following reports 
and recommendations: 

1 . Any post-program measurement information. 

2. Comparisons of pre-program and post-program 
measurement. 

3. Analysis of objective attainment. 

4. Revisions of program variables and sampling 
methods. 



5. Revision of performance objectives and process 
objectives, 

6. Revisions of measurement controls and instru- 
ments. 

7. Program changes. 

8. Recommendations for students not achieving 
performance objectives. 

It is pertinent to consider feedback from the various 
publics and decision makers for application to contin- 
ued assessment and planning. There is no definite ter- 
mination of feedback between steps. Feedback is always 
a vital ingredient to program success. 



RECYCLING 
(Step 7.0) 



This step involves obtaining the agreement of appro- 
priate decision makers to recycle the program with any 
modifications deemed necessary. It is pertinent to 
remember that recycling involves a review of the goals 
and performance objectives plus the needs assessment 
and problem definition steps. While these processes 
should be much easier tO expedite when recycling, they 
are, nonetheless, vital to continued program develop- 
ment. In short, recycling involves thorough review and 
possible revision of every aspect of the system. Too fre- 



quently, recycling is erroneously conceived as simply 
running the old program over again. 

Another graphic illustration of the interrelated steps 
of comprehensive program planning operation and eval- 
uation is depicted in Figure 10. 

Note that feedback is continuous, which is also to say 
that assessment and evaluation are continuous. 

Another version of a linear model which embodies 
the same concept as that in Figure 10 is presented in 
Appendix C, Figure 14. 



CONCLUSION 



The model described herein is not intended to be a 
perfect solution for any and all organizations; rather, 
the suggested elements are ideas for solution strategies. 

Other alternatives must be considered and evaluated 
in terms of feasibility within the resources available, the 
operational and social-political environment and the 
objectives of the planning unit. 



To be effective in planning, some means, structural 
as well as operational, should be established which will 
facilitate the systematic application of intelligence to the 
resolution of problems. 

The challenge of management today is to combine 
democracy and recognition of individual worth with 
management science technology for the development of 
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optimum modes of efficient and relevant services. The 
two must proceed together, and to do so there must be 
a purpose for each. Without purpose there can be no 
point to activity. The better the definition of purpose, 
through goals and objectives, the more distinct and 
revealing the evaluation. When two or more people are 
working together there must be agreement on common 
goals. 

The best type of management would be that type 
which integrates the work of an individual toward the 
overall objectives of an enterprise, while at the same 
time considering the individual's personal interests and 
desires. To accomplish this requires an in-depth defini- 
tion of the goals and objectives at all levels of the enter- 
prise. 

Management by objectives is a planned approach to 
individual performance within an enterprise. It is the 
antithesis of management by reaction, where planning 
(if it occurs) is accomplished immediately prior to or in 
concert with action. In this latter methodology of man- 



agement, activity is measured by the flurry of things 
being done, and the effort that is put forth, rather than 
by the results they produce. In management by objec- 
tives activity is measured by results. 

Educational systems are especially vulnerable to 
intervening social, political, and circumstantial issues 
and needs. This vulnerability makes it imperative to 
build into the operational procedures a reasonable sys- 
tem of priority do*s and don'ts. There must be a plan 
and the will to say we can't do this because there are 
more important priorities. 

In applying this method of management one is con- 
cerned with the correlation between work done and the 
purpose of the enterprise in which it was done. Any 
work which does not contribute in some way to the 
overall goals is either misdirected because of the ineffi- 
ciency of the enterprise to direct it or because someone 
is intentionally directing it away. Failure for the latter 
reason is more forgivable than for the former, and both 
should be eliminated. 
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GLOSSARY OF TERMS 



Analysis The process of determining the 

parts of a structure and the way in 
which they relate, one to the other 
and with the total system. 

Assessment Activities which include objective 

and/or subjective techniques de- 
signed to determine educational 
needs. 

Accountability The responsibility to answer for the 
results of actions and objectives; the 
legal and/or moral responsibility to 
be answerable for the what, why, 
and how of programs, proceedures, 
and products. 

Planning which involves: (1) Con- 
sideration of all relevant factors 
(2) participation of all persons 
who should contribute, and (3) in- 
tense, broad, long and short range 
projection. 

Hurdles, real or imagined, which 
may jeopardize the successful ac- 
complishment of any part of the 
objective. 

To state the meaning of, or to de- 
scribe with clarity in relation to, 
identifiable, tangible, and/or intang- 
ible conditions. 

The distribution of inforrnation 
about a program, project or activity. 

A quantitative and qualitative meas- 
ure which can be used to evaluate 
the level of performance in relation 
to some standard set of criteria. 

Activities undertaken in an attempt 
to determine the valu6 of a pro- 
gram, project, technique, process. 
Of product in relationship to a pre- 
determined objective. 

Feedback The factual and evaluative informa- 

tion which describes the functioning 
of a system and is used as a basis 
for modification of the system. 

Functions Those actions which must be com- 

pleted to accomplish the objective. 

Generic A class of related things having a 

general application. 

Goals Ends toward which an agency or 

person performs work or renders 
services; a statement of direction 
and purpose. A goal is general and 
timeless in that it is not concerned 
with a specific achievement within a 
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specified time period, nor is any 
measurement specified. 

Persons or other entities arranged or 
positioned in a series of ascending 
and descending values. 

To recognize by forms, shape, size 
and/or description. 

To carry out; to apply the time, 
effort, and materials required to get 
the job done. 

A generic pattern which may be ap- 
plied to a specific process or a re- 
lated set of processes. 

The difference between what is and 
what should be. 

Determination of the difference be- 
tween what is and what should be. 

Intermediate ends for which work is 
performed or services rendered; the 
desired product of an activity repre- 
senting progress toward goals. 

One of a set of properties whose 
values determine the characteristics 
or behavior of a system. 

A statement that specifies a desired 
change in behavior on the part of 
the learner; the statement answers 
the following questions: 

1. Who is going to perform 
the specified behavior? 

2. What behavior is expected 
to occur? 

3. What is the expected pro- 
ficiency level? 

4. Under what circumstances 
is the behavior going to be 
observed? 

5. What amount of time or 
necessary prerequisites are 
needed to bring about the 
specified behavior? 

6. How is the behavior going 
to be measured? 

Development of the detailed opera- 
tional guidehnes and criteria for use 
in the implementation of a project 
or program. 

That which is preferred in rank or 
position. 

The recognition of a need which re- 
quires further study. 
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Glossary of Terms— Continued 



Problem 
Definition 

Process 
Objective 



Program 



The delineation of the factors and 
influences which contribute to the 
need. 

A statement that describes an activ- 
ity which affects the performance of 
a learner or results in a desired 
product — it answers the following 
questions: 

1. What specific activity? 

2. Who will be conducting 
the activity? 

3. What time span? 

4. What specific, tangible out- 
come will result? 

5. Cost? 

A major agency endeavor, goal- 
oriented, comprising multiple proj- 
ects and tasks, which is defined in 
terms of the principal actions re- 
quired to achieve specified objec- 
tives. 



Strategy 
Syncretic 



System 

Systematic 
Synthesize 

Tasks 

Variable 



A careful plan or method. 

A harmonious blend of various ele- 
ments and/or functions for pur- 
poses of accomplishing specified 
objectives or producing desired 
products. 

The sum total of parts working in- 
dependently or relatedly to achieve 
specified objectives. 

Acting by a predetermined system 
or comprehensive method. 

To put together in the most relevant 
manner the parts of a system to 
achieve determined objectives. 

Elements of an activity which, when 
completed, will help complete the 
activity. 

An element subject to change. 



APPENDIX A— EDUCATIONAL VARIABLES 



In the process of problem clarification and program 
planning it is important to consider the major educa- 
tional variables and to use them to clarify written 
objectives. This will help to organize the processes and 
to select evaluation instruments. Figure 1 1 displays a 
model of the major variables.^ 

In the institutional dimension, the variables listed are 
familiar. Community may be assumed to encompass 

M Scheme for Evaluation and an Organizational Structure of 
Variables. Tucson, Arizona: Educational Innovators Press, Inc.. 
1970. Reprinted with permission. 



those familiar vpriables such as service clubs, P.T.A., 
religious, politic .l, and similar groups. 

Under the instructional dimension organization might 
mean self-contained classroom, non-graded, depart- 
mentalized or other forms of grouping. Content is a 
body of knowledge topically defined, i.e., reading, 
arithmetic, social studies, algebia. Method refers to 
(1) teaching, i.e., lecture, demonstration; (2) types of 
interaction; and (3) learning principles, i.e., '^cognitive 
field." Facilities means space, equipment, and supplies. 
Cost covers operational expenditures. 
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The behavioral dimension has three variables: cogni- 
tive, affective, and psychomotor. Cognitive variables 
include (1) knowledge, (2) comprehension, (3) appli- 
cation, (4) analysis, (5) synthesis and (6) evaluation, 
in ascending order of difficulty. The levels of the affec- 
tive behavioral variable are: (1) receive, (2) respond, 
(3) value, (4) organization and (5) characterization. 



The levels of the psychomotor behavioral variable are: 
(1) imitation, (2) manipulation, (3) precision, (4) 
articulation and (5) naturalization.^ 

These variables are relevant in the process of devel- 
oping performance and process objectives. 

"Performance and Process Objectives, Tucson, Arizona: Edu- 
cational Innovators Press, Inc., 1970. 
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APPENDIX B— GOALS AND OBJECTIVES 



It is axiomatic that we should know where we are 
headed (goals) and have some mileposts for measuring 
our progress (objectives). Not only should individuals 
know where they are going and how well they are pro- 
gressing, but their plans and progress should be made 
clear for purposes of communications and accountabil- 
ity. While goals are, of necessity, broad and general, 
they are essential to the development of operational 
objectives. They provide a frame of reference within 
which we can specify our objectives. 

GOALS 

A goal is a statement of broad direction, purpose or 
intent. It is an end toward which the agency performs 
work. A goal may he general and timeless in that it may 
not be concerned with a specific achievement within a 
specified time period. A hierarchical relationship is 
shown in Figure 12. 

For example: 

Goal 1. (State) To help every child acquire, 
to the fullest extent possible, mastery of the basic- 
skills in the use of words and numbers. 

Goal 2. (Regional) To provide instructional 
services throughout the region which will assure 
each child opportunity to develop his reading skills 
to his maximum capacity. 

Goal 3. (School District) To provide instruc- 
tional services which will maximize the reading 
level of each and every child in school. 

Goal 4. (School) To assure that each child 
enrolled has a reading level at or above his func- 
tional level of capability. 

In order for a goal structure to effect continuity of 
effort throughout the agencies, it is important that goals 




be related to the area of responsibility of the particular 
organizational level. A state department goal, for exam- 
ple, should encompass state aspirations in a given area. 
The district goal should relate to the same concern, but 
be limited to district areas of responsibility. Likewise, 
from district to local school — a decreasing scope of 
jurisdiction and a concomitant specificity of responsibil- 
ity is necessary. While state goals do not dictate district 
and/or school goals, they are related and should influ- 
ence one another. To go below the school level wiih 
written goals is generally unproductive. The classroom 
teacher adopts the school goals and develops a set of 
related objectives. 

OBJECTIVES 

The system for developing objectives is similar to 
that for the development of goals. Objectives can be 
grouped and arrangH a hierarchy with specific objec- 
tives contributing toward broader objectives. An impor- 
tant characteristic of an objective is that it supports a 
stated goal. The establishment of priorities of objectives 
is important if we are to meet educational needs in an 
orderly fashion. There are two major types of objectives 
appropriate to the activities within a statewide educa- 
tional organization. These are performance objectives 
and process objectives. The significant difference 
between these two lies in the expected tangible out- 
come. The activities of administrators, teachers and 
supervisory personnel call for process objectives. 

A process objective has five basic elements: 

1. The individual and/or groups responsible for 
carrying out the activity. 

2. The activity, related to a tangible outcome. 

3. A time factor. 

4. A tangible outcome. 
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5. A cost estimate. 

PrcKcss objectives are those activities designed to 

influence the perjormance of the learners. 

A performance objective has six basic elements: 

1. The institutional variable (i.e.. third-grade 

pupils). 

A behavioral variable (i.e., comprehension). 
An instructional variable (i.e., reading). 
A method of measurement (i.e., CTBS). 
Time or prerequisite (i.e., 38 weeks). 
A proficiency level (i.e., 10 months average 
increase). 

The hierarchical relationships of performance objec- 
tives is outlined in Figure 2. 

0.1 (State) Upon the completion of the 1971-72 
school year, the third-grade students will display a one- 
year gain in reading comprehension as measured by a 
selected standardized test. 

0.2 (Region) Note: The objective will be the 
same as the Staters except that it may have a higher or 
lower proficiency level, depending upon characteristics 
of children and conditions in the particular geographic 
region. 



0.3 (Di.strict) At the end of the third grade in 
school, elementary pupils in district schools will display 
a 10-months gain in reading comprehension as meas- 
ured by a selected standardized test. (Note: higher or 
lower proficiency level depending upon conditions.) 

0.4 (School) Upon completion of the 197i-72 
school year, the third-grade students will display an 
8-month gain in reading comprehension as measured by 
a selected standardized test. 

0.5 (Classroom) At the end of January. 1972. 
third-grade students will apply selected word attack 
skills with a minimum of 40 percent accuracy as meas- 
ured by a teacher-developed test. (Note: There will be 
several such objectives for reading at the classroom 
level including one for a standardized test.) 

0.6 (Learner) At the end of January, 1^72. 
"Johnny" will be able to read from his third-grade text- 
book with a maximum of two errors in pronunciation 
per full page as measured by an informal reading inven- 
tory. 

With objectives determined at the various levels of 
specificity and measurement of student status, we can 
analyze need. 
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APPENDIX C— A SYNCRETIC LINEAR MODEL 
FOR COMPREHENSIVE PLANNING 



Contemporary problems, iaken in historical pcrspec* 
tive, reveal the criticality of good planning. It is 
revealed in the plights of cities, in the social lag of edu- 
cational institutions and the complex social and political 
problems of our time. 

The model presented herein, while indicating seven 
basic steps (as numbered) also shows other interme- 
diate processes which arc to be considered vital to good 
program design and operation. For example: The estab- 
lishment of goals should include more than verbal dis- 
cussion, important as that is. The goals should be 
written, disseminated and interpreted — after a period of 
discussion. 

After a general needs assessment, the identified nce'ls 
should be listed in priority order; they should be 



analyzed and defined in a sj^tematic fashion; and solu- 
tion strategies should be implemented according to pri- 
orities, time, and resources. High priority problems 
should be assigned the resources necessary ti> reduce the 
need even at the expense of reduced emphasis on lesser 
problems rather than proceeding on a broad front with 
the hope that equal treatment will provide succor to all 
needs. 

Finally, the effects of a treatment program should be 
careft'Hy analyzed before dissemination; that is to s;iy, 
perceptual impressions should not become the guide* 
lines of future operations. All evidence should be evalu- 
ated against the performance objectives. Program 
modification is an important consideration based upon 
the summative evaluation, befuie recycling. 
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